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(57) Abstract: A load testing system (100) for testing a web site or other type of server system (122) uses a thiead architecture 
that reduces die computing resources needed to generate a desired load. The load testing system runs several virtual users (102) on 
one or more clients (104) to simulate user interactions with the server. Each virtual user is executed as a virtual user thread (112) 
under a process on a client computer. Each virtoal user thread itself establishes and supports multiple connections (202) to the web 
site; therefore, an additional tiiread need not be created for each connection. For each connection, the virtual user thread performs a 
sequence of functions in an asynchronous mode to establish and support the connection. If a function of a thread cannot complete 
without blocking, it immediately remms and the calling tfircad switches execution to anoUier task. After the condition causing the 
block has been resolved, the thread can switch back to executing the interrupted task. In this manner, the single thread is able to 
support multiple simultaneous connections. 
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USE OF A SINGLE CONTROL FLOW TO SUPPORT MULTIPLE NETWORK CONNECTIONS FOR SERVER LOAD 

TESHNG 

BadcQround of the Invention 

5 Ffeld of the Invention . 

This invention relates generally to the toad testing of web sites and other multi-user systems, and more partodarty, 
the invention relates to software architectures for reducing a quantity of physical tomputing resources needed to generate a 
desired load. 

10 Description of the Related Art 

One aspect of server management relates to the abifity of a server to handle pealc loads. In the context of web sites, 
for example the number of concurrent users a web site can support depends upon many factors. These factore may Include: 
the complexity of the web site's appBcaOon architectore. the number and type of servers that host the web site, the 
configuration of the servers, and the bandwidth of the oonnection(s) to the web site. 
15 Heavy use, approaching or beyond the practical capacity of a server system, can cause response times to degrade, 

sometimes to the point where the system becomes practica»y InaccessiUe. Up to a certain point as ttie number of users of a 
server system Increases, the system's.perfpmiance may. remain at a relativeiy. constent and aoeeptstite teveL B^ond^lhis 
point, however, increased loads may cause drastic degradations in performance. Accordingly, system administrators often 
only become aware of an overload condition once the condition occurs. 
20 Furthemiore, the resolution of such a problem may require replacement of complete server systems, may require 

changes In oommunicatbns systems, and/or may require the partial or total reworlting of software that supports system. 
Depending upon whether addlttonal equipment Is avaflabte or needs to be ordered, adding servers may take fiom several 
hours to several days or even several weeks. Repairing software problems may take from several hours to several months. 

In order to address these problems, scrftware tools aid services have been devetoped to toad test web sites and 
25 other types of server systems. Examptes of such tools and servtoes Include the LoadRunnei® product of Mercury Interactive 
Corporation, and the associated hosted ActiveTest™ service for load tesfing server systems over the Internet Tliese tools 
and services aOow the perfomiance of a server system to be measured under various load conditions, optionally before live 
deployment In this manner, the pracfical capacities of web sites and other multi-user server systems can be WentilJed In 
advance and compared to trends In actoal use tevels. Accordingly, fiitore oveiloading of a server system can be anttelpated, 
30 and solutions can be Implemented, before a probtem actually occurs. 

One problem with exisfing toad testing methods Is that a significant quantily of computetional resources is typically 
needed to generate an appropriate toad. Fw exampte. to generate the toad needed to appropriately stress a popular e- 
commerce site. It may be necessary use ten or twenty dedicated computere. each of whteh simulates the actions of several 
users. The present inventton seeks to address this probtem. 
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Summary of the Invention 

The present Invention provides an improved software architecture for Emulating the actions of users during server 
load festng. The Invention Involves the use of a virtual user control flow, which is preferably a thread, that handles mulfipte 
concurrent connections to the server system during the load testing process. The use of such virtual user threads sIgnilicanHy 
reduces the number of concurrent threads needed to produce a desired load. As a result a given computer can generate a 
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greater bad (simulate a gieater nmber of users) than is possible with conventional methods. The computing resources 
needed to load test a server system are therefore reduced. 

The Invention is preferably embodied within a load testing tool (computer program) that may be made available to 
web site operators as an installable software product, as a hosted load testing service, or both. The load testing tool 
5 preferably runs several virtual users on one or more clients to simulate user interactions with a web site. Each virtual user Is 
executed as a virtual user thread under a process on a client computer. Each virtual user thread itself establishes and 
supports multiple connections to the web site. Additional threads therefore need not be created for each connection. 

For each connection, the virtual user thread performs a sequence of functions to establish and support the 
connection. Functions that may potentially block are executed in an asynchronous mode. When executed in asynchronous 
10 mode, if a function cannot complete without blocking, it Immediately returns a RESOURCE UNAVAILABLE error code. If a 
fiincBon returns a RESOURCE UNAVAILABLE code, the calling thread switches execution to another task. After the conditfon 
causing the RESOURCE UNAVAILABLE error code has been resolved, the thread can switch back to executing the 
intemjpted task. In this manner, the single virtual user thread Is able to support multiple simultaneous connections. 

In an alternative embodiment, the virtual user flow ofoontrol may be embodied as a process rather than a thread. In 
15 this case, the main flow of control supports the mulfiple simultaneous connections. 

The Invention may be advantageously used to deaease the computing resources needed to bad test web sites and 
other types of server systems. 

One embodiment of the Invention Is a method of testing the load handling capability of a server system. The method 
is perfbmied on a client computer. The method includes, within a single control flow, concurrently perfomiing a plurality of 
20 tasks. Each task includes establishing a connecBon to the server system, sending a request through the connection, and 
receiving a response through the connection. The method also Includes, for each task, monitoring an elapsed time between 
the sending of the request and the receipt of the response. 

One embodiment of the Inventton is a software module embodied within a computer readable medium. The software 
module Includes executable code configured to be executed by a computer. The software module is configured to spawn a 
25 thread that submits user requests to a server system through multiple concunrent connections such that multiple user requests 
are pending concun-entiy. The software module is also configured to monitor responses from the server system to the multiple, 
user requests through the multiple connections to monitor pertbmiance of the server system. 

One embodiment of the invention is a method of implementing multiple concurent connectfons through one control 
flow on a client computer. The method is performed on the client computer. The method Includes identifying a plurality of 
30 . tasks, wherein each task is performed through at least one associated function through whteh some funcfionaffty Is requested. 
Each task includes estebDshing a connectton to a server system, sending a request through the connection, and receiving a 
response Birough the connection. The mefliod also Includes, for each task, initiating ttie associated functfons for the task, 
wherein the functions are Initiated in an asynchronous mode and In order until one of the following occurs: a RESOURCE 
UNAVAILABLE code Is returned by a function, and the task completes. The meUiod also Includes, for each task, if a 
35 RESOURCE UNAVAILABLE code is received from a function, saving state Information ttial is descriptive of the state of the 
associated task at ttie point at which ttie RESOURCE UNAVAILABLE code is received. 

One embodiment of the Invention is a virtual user for simulating, in a server load testing system, the actions of a 
client The virtual user Includes a control flow; and 

a Rlurafity of concunent connections to a server system, wherein ttie plurality of connections are estebDshed by the control 
40 flow. 
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One embodlmeni of the Invention Is a system for load testing servers. The system Includes at least one dlent 
computer connected to at least one server under test through a computer network. The system also Includes a control flow 
executing on the at least one dlent computer. The system also Indudes a phjraflty of concurrent connections to at least one 
server, wherein the plurality of connections are established by the control flow. 

5 

Brief Descfjption of the Drawings 
A preferred embodiment of the present Invention will be described belovtf In connection with the drawings in which: 
Figure 1 A illustrates an example configuration of a web site being tested by a load testing system; 
Rgure IB illuslrates the several virtual users executing under a dient process on a dlent computer, 
10 Figure 2 illustrates the connections through which a virtual user communicates with a server, 

Figure 3 illustrates a known meihod performed by the dlent process during a load test of a sender; 
Rgure 4 illustrates the set of stages perfonned by each task thread In handling each task; 
Rgure 5 illustrates a prefened embodiment of the present invention in which a virtual user Siread Itself handles 
requests and supports two or more connections to a server computer; 
15 Rgure 6 illustrates a preferred method for supporting multiple simultaneous connections between a dient computer 

and one or more servers with a single thread; Figure 7 illustrates a preferred method appficable to Unix and Windows 
Sockets Implementatlqns for oonqurrentfy earring ouj[ttie stegas.of a plurglityof.tasks; and 

Rgure 8 illustrates a preferred method applicable to the Microsoft Windows Win32 Internet API (Winlnet) for 
concunently carrying out the stages of a plurality of tesks. 

20 

Detstfled Description of fte Embodiments 
In the following description, reference Is made to the accompanying drawings, which fonn a part hereof, and which 
show, by way of Illustration, spedfic embodimenb or methods In which the invention may be practiced. Where possibte, the 
same reference numbers are used throughout the drawings to refer to the same or like components. In some Instances, 
25 numerous spedfic details are set forth in order to provide a thorough understanding of the present invention. The present 
Invention, however, may be praciioed without the specific details or with certain alternative equivalent components and 
methods to those described herein. In other instances, well-known methods and components have not been described In 
detail so as not to unnecessarily obscure aspeds of the present invention. 

For purposes of illustration, the Invention will be described primarily in the context of the load testing of web sites. If 
30 win be recognized, however, that the InvenJon Is also applicaWe to load teste of other types of multi-user server systems. 
Unless hdtoated othenrtse, ft should be assumed ftat the functions set forth herein are perfonned using software that runs on 
a general purpose computer, whfeh Is connected to the server system to be tested through a computer networic. 
I. PRIOR ART SOFTWARE ARCHITEGTURES FOR SERVER LOAD TESTING 

To fadlitate an understanding of Oie Invention and the problems to which it Is directed, conventional software tools 
35 and metfiods for toad testing servers will initially be descn'bed witii reference to Rgures 1 through 4. These tools and methods 
are generally described in the context of load testing web sites, but may be used to test other types of client/server system. 

Rgure 1A illustrates an example configuration of a web site 120 being tested by a load testing tool or system 100. 
This drawing and the associated description below is representative of bott> prior art load testing tools and tools that operate 
according to the present invention. The web site 120 is typically hosted by one or more sen/ers 122. The toad testing system 
40 100 uses a virtual user component 102, or "Vuser/ that simulates a dlent program's interaction with a web site 120 during a 
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user browsing session. Each Vuser 102 sends requests to the web site 120 according to a pre-defined test script (Vuser 
scripQ 108. The script 108, which may be different for different Vusers. may be in the fonm of a list of the hypertext transfer 
protocol (HTTP) requests to be sent to the web server. The script may also specify the content of expected server responses. 
The script may be read fix)m a script file by the Vuser during execution, or may be compiled within the executable Vuser code. 
5 Each request may also be a secure HTTP or HTTPS request a file transfer protocol (FTP) request, or any type of request that 
may be handled by a server. 

Each script 108 typically specifies a sequence of user actions for performing a particular transaction. For example, 
In the context of a travel reservation web site, a script may specify a search for a particular flight followed by the placement of 
a reservation for that flight A Vuser 102 may be configured, via the user interface of a controller 106, to execute or "pla/ the 

10 script repeatedfy a desired number of times (e.g., 50 iterations) during the execution of a load test Preferably, the Vuser plays 
the script at a rate that Is faster than the "real flme" rate at which a user typlcaify browses the web site, and thus produces a 
toad representaflve of many oonourrent users. 

Several Vusers 102 are typically run on one or more dlent computers 104underthecontrolofthe controller 106. In 
Implementations In which the foad testing tool is provided as a hosted Internet service, each client computer may be a 

15 decficated machine that is locally comiected to an Internet badcbone. In fypical test sHuafions, many hundreds or thousands of 
Vusers 1 02 are mn concurrently to produce a load representaflve of many tens of fltousands of concurrent users. 

During fte load test, each Vuser 102 monitors *e web site^s response-times to-iequests. These measured 
response times serve as indicators of the web site's performance, and can be aggregated and analyzed to create performance 
data. The perfonnance data Is fypically presented to tfie user of the load testing tool or service through a series of predefined 

20 graphs and charts. Additional details of commercially available tools for load testing server systems are described, for 
example, In U.S. Appl. No. 09/337,446, filed June 21. 1999, the disclosure of which Is hereby incorporated by reference. 

Under actual, non-test conditions, each client computer 104 typically only executes one web browser at a time. In ' 
ottier words, only one person can typically browse ttie web on one computer at one time. In a test configuration, however, 
there is fittie or no advantage to executing only one Vuser 102 on each client computer 104 - Indeed this would be an 

25 Inefficient use of resources. Each dlent computer 104 in a test configuration will typlcaOy have tiie compute power and ttie 
communications bandvwdtti necessary to support several hundred or even several thousands of Vusers 102. A load testing 
system 100, tiierefore, typically mns multipte Vusers on each client computer 104 to create ttie desired load. 

A flow of control (control flow), as used herein, refers to tiie execution of a sequence of Instnjctions of an application 
running under an operating system. In any single flow of control, tiie relative order in which tiie instmctions are' executed is 

30 determbied by ttie Instructions tfiemselves as opposed to tiie operating system. 

A process executing under an operating system generally includes a main flow of control as well as certain system 
resources, such as a block of virtual memory, IMost operating systems support mulfitasldng of processes (multitasking 
operating systems). Multitasldng allows a system to switch execution among processes such that each of several processes is 
effectively being performed concurrenfly by tiie system. Processes, however, are generally an expensive system resource. 

35 Most operating systems today also support ttie use of ttireads (mulflflireaded operating systems). A parent process 

can create one or more associated ttireads. Each ttiread represents a separate flow of control vrtfliln its parent process. A 
tfiread, however, is a less expensive system resource flian a process since a thread generally utilizes syslem resources 
already allocated to its parent process, such as tfie virtual memory space of tiie parent process. Several tiireads can be 
spavmed wittiln a process to Implement concurrent control flows vwtiiin a singte process. 
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As Illustrated In Figure IB, In order to conserve system rssouroes, sevefBl Vusers are typically m under one client 

process 1 10 as opposed to running each Vuser as a separate process. The main flow of control for each Vuser Is typically a 

Vuser thread 112, which, although still an expensWe system resource, Is less expensive than a prooess. 

Figure 2 Illustrates the connections 202A-D through which a Vuser 102 communicates with a web server. Each time 
5 an HTTP request Is made, whether by a real user browsing the web through a web browser or by a Vuser 102, the request Is 
sent and a response received through a connection to the web server. HTTP requests generally take some time to be fulfilled, 
ranging from a fraction of a second to several seconds or even minutes. Each request also typically requires its own flow of 
control and can oftentimes be a bloclcing (stalling) task, in order to Increase data throughput and decrease latency, a Vuser 
102 (or a web browser) typically generates several simultaneously outstanding requests. Since each request Is generally a 
10 blocking task, the Vuser thread 112 spdms a new thread 210A to handle the request The thread 210A typically generates 
the request and opens the connection 202A. The thread 210A pereists unfit the response is received and the connection 202A 
ck)sed, at which time the thread 210A is destroyed. 

Vusere 102 (or web browsers) are typically configured to altow up to four simultaneous connecfions to support up to 
four outstanding requests, however this number can be adjusted. Once there are four outstanding requests, addifional 
15 requests are deferred until one of the outstanding requests has been satisfied. Vusers 102, in order to properly simulate real 
users using web browsers, are also typically configured to support up to four simultaneous outstanding requests 202A-D, each 
supported by an associated thread 21QA-D. 

When a client computer Is executing a single web browser, the additional four threads present an inconsequential 
demand on flie system. A load testing host, however, may be mnning 1000 Vusere 102, and if each Vuser 102. has 4 
20 additional threads, file system has to support an addifional 4000 threads. When used in such large numbere, the additional 
overhead necessaiy to support these additional threads becomes a substantial burden on fiie system. 

Figure 3 illusfrates a known method 300 performed by fiie cBent process 110 during a foad test of a server 122. At a 
step 302, the client process 1 1 0 creates a plurafity of Vusere 10Z EaA Vuser 102 is typically embodied as a separate Vuser 
thread 112. The functionality of each Vuser 102 is typically specified by a Vuser script 108. 
25 At a step 304, a Vuser thread 112 generates one or more tasks based upon the Vuser script 108. Each task defines 

an exchange of data between the client computer 104 and a web server. In the case that a web server 122 is being foad 
tested, ttie tesk typlcaHy defines an HTTP request specified in the Vuser script 108. The HHP request specifies the metiiod 
{e.g., GET or POST), the uniform resouroe locator (URL), and ttie request body or date of the request The request may 
alternatively be a secure HTTP (HTTPS) request, a fite transfer protocol (FTP) request, or any type of request that may be 
30 handled by a server. AW of the VUser threads 112 created In the step 302 may perfomi the step 304 concurrently and 
repeatedly as necessary. 

At a step 306, a Vuser thread 1 12 creates a new task fliread 210A to handte each task. The newly created ttvead 
210A results in a separate flow of control In which file task Is perfomned. as illustreted by fiie arrow fliat leads to a step 308. 
The Vuser ttiread 112 Is also free to continue witii ite own flow of control and possibly create addifional tesks and threads 
35 201 B-D. as Illustrated by the arrow tfiat loops back to flie step 304. 

At file step 308, ttie tesk ttiread 210A perfonns a set of steges 400 to complete ttie task. The set of stages 400 is 
niustrated in Figure 4 and described below. At a step 310, once fiie tesk has been successfully completed, the tesk fliread 
210A exite. If file maximum number of connecfions permitted by flie Vuser had been outstanding prior to the step 310. ttie 
Vuser may now create an addifional fiiread to handle anottier task In accordance witti flie steps 304 and 306. 
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Rgure 4 illustrates Sie set of stages 400 performed by each task thread 210 in handling each task during a load test 
of a web server 122. The set of stipes 400 and Oieir Implementation will be familiar to one skilled in the art Several of the 
stages are potentially blocking, which means that the stage may Invoha an unpredictable delay before compleOon. 

At a first stage 402, the thread 210 parses the UFO. into its components. The components may include the host 
name, flie port, flie path, etc. This stage typkally Is non-bkxking and wtD not cause a wait At a stage 404. the thread 210 
resolvM the host name into an IP address. Typkally, the stsge 404 m|uires the querying of a name server, and therefore Uds 
stage may block or cause a wait 

At a stage 406. the thread 210 establishes a connection to the server 122, which can cause the thread to block. At a 
step 408. the thread 210 generates a request which preferably includes headers and a body. At a stage 410. the thread 210 
issues (sends) the request which can cause the thread 210 to block, in the case the request is tong, the request may be 
transmHlBd h multiple segnients. 

At a stage 412, the thread 210 reads a response segment which is relumed by the server 122. The stage 41 2 can 
cause Oie thread 21 0 to bkxsk. At a stage 414, If Ihers is more data expected or to be r«ceh^, the thread lepeab the step 
412 as necessary. If the ttiread 210 has completed reading the data, it proceeds to the stage 416. At the stage 416, fiie 
15 thread 210 dom the oonnectkm, and at a stago 418, the thread 210 fiees any unnecessary resources. 

Each stage is typically carried out by calling one or more ftmctlons. Each fiincBon call represents a request that 
certain fijncHonality be perfom^^^ Many of the functtons are operating system ftmcitons, in whki) case the.opera8ng.6ystem 
generally carries out the requested fiincBonaiify. Some of the functions, however, may be supplied by the programmer. In this 
case, the supplied function itself may carry out the requested funcBonalily. Alternatively, the supplied flinctton may be 
20 configured to call other functions, possibly Including operating system functions, to cany out the requested functionality. The 
programmer may also choose to write his own verskms of some of ttie operating system supplied functions. 

Severd of the fiinctions used to cany out the potentially blocking stages 404-416 will btock upon being called if the 
funcBonaiity to be perfonned cannot completed Immediately. A potenfialy blocking function generally bhicks because a 
required resource is not available at the time the lunctkm is called. If a fimction blocks, the thread 8)at called the functkm 
25 cannot proceed until the resource becomes availabte and the called lunctton returns, if the resource does not become 
available within a certain amount of time, the fiincfion may time out and return an enor code. Since each request is handled 
by a separate thread, eadi thread can safely btock without affecting concurrent executton of the Vuser thread 112 or the other 
task threads 210. 

Functtons that may potentially block typically also support an asynchronous mode. When a function is executed In 
30 an asynchronous mode, the luncCon will always return wilhout blocking, regardless of whether the requested funcUonality has 
been completed, if the requested functionality has been corqpleted, the called function generally returns a code Indicating 
successful GompleBon. if Uie requested functlonali^ has not been oompteted. however, the caUed iiincBon wfll return an error 
code. In some cases the error code indicates Oiat the requested functionality cannot be performed due to a fetal error. 

When the funcfions are called in asynchronous mode, however, most error codes are of a ^ hereinafter referred 
35 to in general as RESOURCE UNAVAILABLE error codes. A RESOURCE UNAVAILABI^eiTor code indkates that a required 
resource is not available at the Hme the ftinclion is cafled and perfomiing the requested functfonaOfy wouM cause the calling 
thread to block. As a result the caHed function returns Immediately with a RESOURCE Ui^VAlLABLE error code. In some 
implementations, such as Unbc and Windows Sockets, the called function does not perform the requested fijnclionality and 
must be called again after retuming the RESOURCE UNAVAILABLE code. Typtealiy, upon being called again, the function 
eventually returns a code indicating successful completion. In other implementetfons such as the H^iorosoft Windows Win32 
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Internet API (WinlneQ, the caOed function causes the requested functionality to be perfbmied in the background after returning 
the RESOURCE UNAVAILABI£ error code. The calBng thread is eventually notitied of completion of Oie requested 
fimcOonality through the calOng of a predefined callback fiindioa In these implementations, the function needs to be called 
only once. 

5 The RESOURCE UNAVAILABLE error code Is typically a rtamed constant and Is typically specified in a program 

code header file. Tlie name of a RESOURCE UNAVAILABLE error code typically depends upon the implementalion used. 
For example, in Unix Sockets Implementations, the RESOURCE UNAVAILABLE code Is typkally called EWOULDBLOCK or 
EAGAIN. In the Windows Sockets (WinSock) implementations, the RESOURCE UNAVAILABLE code Is typfcally called 
WSAEWOULDBLOCK. In the Wmlnet environment, the RESOURCE UNAVAILABLE code is lyplcally caDed 
to ERROR_IO_PENDING. 

II. MODIFIED LOAD TESTING TOa AND ARCHITECTURE FOR SUPPORTING MULTIPLE CONNECTIONS 
THROUGH A SINGLE THREAD 

A. Overview of Components 

Figure 5 illustrates a preferred embodiment of the present Inventkm in which a Vuser thread 112 itself supports two 
15 ormore connecifems 202 to a server computer 122. Instead of creating a new thread to handle each new request one ftread. 
preferably the Vuser ftread. is used to handle multfde outstanding requests simultaneous^ so m no new threads need be 
created. The system Is therefor© not buixtened wiBi ttie expense and overt^ The ieducttonof the 

required system resources for simulating Vusers altows additional Vusers to be run on each cfient computer. As a result, a 
higher simulated load can be aeated using the same amount of computing resources. AltemaOvely, a test can be cojiducted 
20 using fewer computing resources. 

B. General Method 

Figure 6 IHustrates a general method 600 Ibr supporting multiple simultaneous connectfons between a client 
computer and one or more servers within a single thread. The single thread is preferably a Vuser thread 1 12. Several Vuser 
threads 112, however, are pretsrably created through the method 600, and each Vuser thread 112 supports mulOple 
25 simultaneous connectfons. 

In the preferred embodiment, each connection is in the form of a soctet that Is opened udng he TCP/IP protocol. 
Typically, the socket is closed once the server has responded to a user request The Vuser test scripts may, howrever. include 
standard HTTP commands that instruct the server to keep a socket open. 

At a step 602, the client process 110 creates one or more Vusers 102 and preferably several Vusers ^0Z Each 
VUser 102 is preferably embodied as a separate Vbser thread 112. The functionality of each Vuser 102 is preferably spedfied 
by a Wiser script 108 as set forth above. The remaining steps of the method 600 are preferably performed for each Vuser 102 
that has been created. The step 602 may be repeated as necessary to create addlUonal Vuser threads 1 1 2. 

At a step 604. a VUser thread 1 12 generates one or more tasks based upon the Vkiser script 1 08. Each task ddlHies 
an exchange of data behveen the dient computer 104 and a server. In the case that a web server 122 is being load tested, 
the task typically defines an HTTP request spedfied in the Viiser script 108. The HTTP request spediies the method (ag., 
GET or POST), the uniform resource locator (URL), and the request body or date of the request The request may 
alternatively be an HTTPS request, an RP request or any type of request that may be handled by a server. AO of the Vkiser 
threads 1 12 created in the step 602 may perfonn the step 604 concurrently and repeatedly as necessary. 



30 
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At a step 606. the Vuser thread 112 includes in a data stmclure an Identification of each of the tasks to be handled 
concurrently. The data structure is preferably a linked list, but may be any data structure capable of holding or Identifying 
(asks. The steps 604 and 606 are preferably, but need not be, performed together. 

At a step 608, the Vuser thread 112 carries out the stages of each task. The functions of each stage are carried out 
5 asynchronously so that the functions do not cause the thread to block. If a fiinclton returns a RESOURCE UNAVAILABLE 
error code, the thread switches execution to another task. After fte condltton causing the RESOURCE UNAVAILABI^ error 
code has been resolved, the calling thread can switch back to executing the intemjpted task. The Vuser thread 112 preferably 
uses the data structure to keep track of the tasks being canying out In this manner, the single Vuser thread 112 is able to 
support muiOpie 8inf)ultaneous connections. 
10 At a step 610. the Vuser thread 112 monitors the performance of the stages of each task. In the prefen-ed 

embodiment the Vuser thread 112 monitors the elapsed lime between the sending of the request and the receipt of the 
response. The step 610 is preferably performed with ttie step 608. 

At a step 612, the client process 110 or the controller 106 preferably aggregates the perfonmanoe date collected by 
each Vuser thread 112. The date can then be analyzed to detemiine system performance. 
15 The foBowing subsections describe preferred methods for concurrently carrying out the steges a plurality of tasks in 

accordance with the step 608. 

C. - Sod(ete:lmptementaitons 

Figure 7 illustrates a preferred method 700 applicable to Sockete implementaUons. such as Unix and Windows, for 
concurrently carrying out the stages of a plurality of tesks. In Sockets implementetions. although the requested functionality is 
20 not perfonDed when a RESOURCE UNAVAILABLE code is received, a function can again be called successfully when the 
required resources become availabte. This asynchronous mode functionality is applied in the method 700. 

At a step 702, the Vuser thread 112 performs as many stages as possibfe for a task In the date structure. The 
functions of each stege are preferably calted In sequence until either a RESOURCE UNAVAILABLE retum code is recehred or 
the task completes. Potentially btocking functions are executed In an asynchronous mode. 
25 At a step 704. if a RESOURCE UNAVAILABLE code Is received from a funcfion, the Vuser thread 1 12 saves stete 

information that describes the stete of the assodaled task at the point the RESOURCE UNAVAILABLE code Is received. The 
state information allows the remainder of the tesk to be completed when execution of the task Is resumed. The Vtiser thread 
1 12 also preferably identifies the associated tesk as "awaiting resources." 

At a step 706, the Vuser thread 112 repeats the steps 702 and 704 for the remaining tasks in the date structure, 
30 At a step 708, the Vuser thread 112 detennines whether there are any tesks that have received a RESOURCE 

UNAVAILABLE retem code and have yet to complete. If so, the VUser thread 112 passes control to a step 710. If not the 
Vuser thread exite the method 700. 

At the step 710, the Vuser thread 1 12 waits for resources to become available for at feast one of ttie tasks Identified 
as -awalfing resources." The Vuser thread 112 also Identifies the tasks for which resources have become avaUable. if 
35 resources are availabte immediately, the Vuser thread 1 1 2 need not wait 

In a Sockete Imptementetion, the Vuser thread 112 preferably perfomns the step 710 by calling the "select" Sockete 
function call. The Vuser thread 112 identifies in the function caD all of the resources for which all of the tasks are waiting. The 
select function call returns as soon as any of the resources become available or when a timeout expires. Upon retum of the 
"select* call, the Vuser thread 112 preferably calls the "Is.sef function to determine for which tasks resources have become 
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available. The "is.sel" function may also be called before the "selecT function call, and if resources are available for a task, 
the 'selecf call need not be made. 

At a step 712, the Viiser thread 112 repeats the steps 702 and 704 for the remaining stages of the task(5) id^tiiled 
In the step 710. For each task, the saved state infonnation is restored and the stages of the task are continued with the 
5 funcUon that returned the last RESOURCE UNAVAILABLE reium code. 

After the step 712, the Vuser thread 112 returns control to the step 708 to process the remaining tasks until ail of the 
tasks have been completed. 

D. WInlnet Implementation 

Figure 8 illustrates a preferred method 800 applicable to the Microsoft Windows Win32 Internet API (WInlnet) for 
10 concun-ently carrying out the stages of a plurality of tasks. In WInlnet implementations, although a function may return 
immediately with a RESOURCE UNAVAILABLE (ERRORJO^PENDING) code, the requested functionaiity is still performed in 
the background by the WinlneL The calling fiiread Is eventually notilied of completton of the requested fiincfionality through a 
predefined callback function. The function, therefore, needs to be called only once. 

At a step 802, the Vuser thread 112 perfbnns as many stages as possible for a task in the data structure. The 
15 fimcBons of each stage are preferably' performed In sequence until either a RESOURCE UNAVAILABLE return code is 
received or the task completes. Functbns are executed In an asynchronous mode by specifying asynchronous mode upon 
iniSdii^tipn.ofyyinlnet 

At a step 804, if a RESOURCE UNAVAILABLE code Is received from a function, the Vuser thread 112 saves state 
infomialion that describes the slate of the associated task at the point the RESOURCE UNAVAILABLE code was received, 
20 The state infomriatlon allows the remainder of the task to be completed upon completion of the requested functionality. 

At a step 806. the Vuser thread 112 repeats the steps 802 and 804 for all of the tasks in the data structure. 

At a step 808, the Vuser thread 112 detemiines whether there are any tasks that have reodved a RESOURCE 
UNAVAILABLE return code and have yet to complete. If so, the Vuser thread 112 passes control to a step 810. If not. the 
Vuser thread exits the metf)od 800. 
25 At the step 810, the Vuser thread 112 waits until it receives notification that some fiincBonality, which was requested 

through one of the functions that returned RESOURCE UNAVAILABLE, has been completed. The calling ftread is preferably 
notified of the completion through a predefined callback function. 

At a step 81 2, the Vuser thread 1 12 repeats the steps 802 and 804 for the task for which notification was received In 
the step 810. The stages of the task are continued from the point at which the last RESOURCE UNAVAILABLE return code 
30 was receh^ for tfie task. 

After the step 812, the Vtjser thread 112 returns control to the step 808 to process the remaning tasks until all of the 
tasks have been completed. 

III. APPLICABILITY, ALTERNATIVE EIWBODIMENTS AND ADDITIONAL FEATURES 

The software architecture and methods set forth above may be incorporated Into existing load testing tools and 
35 services, such as the LoadRunnertS program and associated AcbVeTest™ service of Mercury Interactive Corporafion. 
AddlHonai details of LoadRunner and ActiveTest are set forth In U.S. Patent 5,974,572. and U.S application number 
09/484,684, filed on January 17. 2000, the disclosures of which are hereby incorporated by reference. 

It will be apparent to one sklOed In the art that various modifications and alternations can be made to the methods 
600, 700, and 800 within the scope of the Invention. For example, tasks can be removed from the data stmcture as they are 
40 completed. New tasks can also be added to the data structure as completed ones are removed. 
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The present invention is particulaily applicable to modem multithreaded operating systems, in this context, several 
Vusers 102 can aiso be executed under a single thread. The Invention may, however, be applied in the context of processes 
Oiat do not include multiple threads. For example, each Vuser 102 may be executed as a separate process rather than as a 
separate thread within a process. The methods 700 or 800 can then be perfomied by the process Itself. 
5 A number of additional features can be added to the methods 600, 700, and 800 to furttier increase ttie performance 

of a load testing system. In one embodiment, a oonnectton can be maintained after a request has completed. The connection 
can then be used for additional requests. In one embodiment, the results of performing certain stages for one tasic may be 
reused to perform the same stages for other tasks. For example, In most cases a hostname need only be resolved Into an IP 
address one t'me. Thereafter, the resolved IP address can be cached and used whenever the same hostname appears again. 
10 IV. CONCLUSION 

Although the Invention has been described In terms of certain preferred embodiments, other embodiments that are 
apparent to those of ordinary skill in the art, including embodiments which do not provMe all of the features and advantages 
set forth herein, are also within the scope of this inventton. Aooordingly, the scope of ttie invention Is defined by the claims that 
follow. In the method claims, reference characters are used for convenience of description only, and do not indteate a 
15 particular order for perfomiing the method. 
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WHAT IS CLAIMED IS: 

1. A method of testihg the load handOng capabflity of a server system, the method comprising, on a dlenl 

computer 

(A) within a single control flow, concurrently performing a plurality of tasks, wherein each task comprises: 

establishing a connection to the server system, 
sending a request through the connection, and 
receiving a response through the connection; and 

(B) for each task, monitoring an elapsed fime behveen the sending of the request and the receipt of the 

response. 

2. The method of Claim 1. further comprising, on the client computer, collecting the monitored elapsed times 
for the plurality of tasks. 

3. The method of Claim 1, further comprising, on the client computer, creating a plurality of virtual users, 
wherein each virtual user performs (A). 

4. The method of Qalm 3, wherein each virtual user is configured to simulate the interactions of a client 
program with the server system. 

5. The method of Claim 1 . wherein the control ftow Is a thread. 

6- The method of Claim 1, wherBin the control flow is a process comprising fewer threads than the number of 
concunrent connections. 

7. The method of Claim 1 , wherein the control flow is the main control flow of a process. 

8. The method of Claim 1 , wherein (A) comprises: 

(A-1) for each task, initiating functions to perform the task, wherein the functions are initiated in an 
asynchronous mode and In order until one of the following occurs: 

a RESOURCE UNAVAIlj\BLE oode is returned by a function, and 
the task completes; and 

(A-2) for each task, if a RESOURCE UNAVAILABLE code is received from a function, saving state 
information ttiat is descriptive of tiie state of the associated task at the pdnt at which the RESOURCE 
UNAVAILABLE code is received. 

9. The metiiod of Claim 8, wherein (A) furtiier comprises: 

{A-3) receiving notiffcation tiiat functionality requested through a function has completed; and 
{A-4) performing (A-1) and (A-2) for the task for which notification was received in (A-3) for any remaining 
functions from the point at whfch ttie RESOURCE UNAVAILABLE codte Is receded. 

10. The mefliod of Claim 9. wherein (A) further comprises 



11. A sofhvare module embodied within a computer readable medium, flie software module comprising 
executable oode that, when executed by a computer 

spawns a ttiread tiiat submits user requests to a server system ttirough multiple concurrent connections 
such tiiat multiple user requests are pending concun^tiy; and 

monitors responses from ttie server system to ttie multiple user requests ttirough ttie multiple connections 
to monitor perfonnance of ttie server ^em. 
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12. The software module as In Claim 11, wherein the thread concurrently performs multiple tasks that 
correspond to the multiple user requests, and switches between Individual tasks of the multiple tasks In response to 
notifications of resource availaMity conditions. 

1 3. The software module as In Claim 12, wherein the thread perfomis stages of each task asynchronously, 

14. The software module as In Claim 11, wherein at least some of (he user requests are HTTP requests 
directed to a web site. 

15. The software module of Claim 1 1 , wherein the executable code Is configured to: 

(A) perform a plurality of tasks, wherein each task is performed through at least one associated 
function through which functionality is requested, and wherein performing each task comprises establishing a 
connection to the server system, sending a request through the connection, and receiving a response from the 
server system through the oonnectlon; 

(B) Initiate, for each task, the associated functions for the task, wherein the functions are initiated in 
an asynchronous mode and In order until one of the following occurs: 

a RESOURCE UNAVAILABLE code is returned by a funcOon. and 
the task completes; and 

(C) for each task, if a RESOURCE UNAVAILABLE code is receWed from a function, save state 
information that is descri|)tiye of tbe.state of the assodated. at which the RESOURCE UNAVAILABLE 
code Is received. 

16. The software module of Claim 15, wherein the executable code is configured to: 

(D) receive notification that functionality requested through a function has completed; and 

(E) perform (B) and (C) for the task for which notification was received in (E) for any remaining 
functions from the point at whteh the RESOURCE UNAVAILABLE code Is received. 

1 7. The software module of aalm 16, wherein the executable code Is further configured to: 

(F) repeat (D) and (E) until an of the tasks have been completed. 

1 8. The software module of Claim 17, wherein the executable code is configured to: 
create a plurality of virtual user threads; and 

for each virtual user thread, perfomn (A), (B), (C), (D), (E), and (1^. 

19. The software module of Claim 16, wherein the executable code is configured to: 

(D) wait for resources to become available for any task for which a RESOURCE UNAVAILABLE 
code has been receh^ed; and 

(E) for a task for which resources have become available in (D), perform (B) and (C) beginning with 
the function that returned the RESOURCE UNAVAILABLE code. 

20. The software module of Claim 1 9, wherein the executable code Is further configured to: 

(F) repeat (D) and (E) until afl of the tasks have been completed. 

21 . The software module of Claim 20, wherein the executable code is further configured to perform (P^, (B), 
(C). (D), (E), and (1^ for each of a plurality of threads. 

22. A method of implementing multiple concunent connections through one control flow on a client computer, 
the method comprising, on the client computer 

(A) identifying a plurality of tasks, wherein each task is perfbmied through at least one associated function 
through which some functionality is requested, and wherein performing each task comprises: 
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establishing a connedion to a server system, 
sending a request through the oonnectioni and 
recehnng a response through the connection; 

(B) for each initiating the associated ftincGons for the task, wherein the funcBons are initiated in an 
asynchronous mode and in order until one of the following occurs: 

a RESOURCE UNAVAILABLE code is returned by a function, and 
the lasic completes; and 

(C) for each task, if a RESOURCE UNAVAILABLE code is received from a function, saving state 
information that is descnptlve of the state of the associated task at the point at which the RESOURCE 
UNAVAILABLE code is received. 

23. The method of Claim 22, further comprising: 

(D) receiving notifkation that functionality requested through a function has completed; and 

(E) perfonning (B) and (C) for the task for whteh nofificatfon was receh^ed In (E) for any remaining functions 
from the point at which the RESOURCE UNAVAILABLE code Is received. 

24. The method of Gaim 23, further comprising 

(F) repeating (D) and (E) until all of the tasks have been completed. 

25. ThemethodvOtClalm24,:furttier.comprlslng: 

(G) creating a plurality of virtual user control flows; and 

(H) for each virtual user control flow, performing (A), (B). (C). (D), (E). and (F). 

26. The method of Claim 22, further comprising: 

(D) waiting for resources to become available for any task for which a RESOURCE UNAVAILABLE code 
has been receded; and 

(E) for a task for which resources have become available in (D), performing (B) and (C) beginning with the 
function that retumed the RESOURCE UNAVAILABLE code. 

27. The method of aakn 26, further comprising 

(F) repeating (D) and (E) untfl all of the tasks have been completed. 

28. The method of Qaim 27. forther comprising: 

(G) creating a plurality of virtual user control flows; and 

(H) for each virtual user control flow, performing (A), (B). (C). (D), (E). and (F). 

29. The method of Oaim 22, forther comprising: 

(D) creating a plurality of virtual user control flows; and 

(E) for each virtual user control flow, perfomAig (A), (B), and (C). 

30. The method of Claim 29. wherein each control ffow is a Biredd. 

31 . The method of Claim 29. wherein each control ffow is a process comprising fewer threads than the number 
of concurrent connections. 

32. The method of Claim 29. wherein each control ffow is the main control flow of a process. 

33. A virtual user for simulating, in a server load testing system, the actions of a dient, the virtual user 
comprising: 

a control flow; and 
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a plurality of concurrent connections to a server system, wherein the pIuraGty of connections are 
established by the control flow. 

34. The virtual user of aabn 33, wherein the control flow is a thread. 

35. The virtual user of Cl^m 33, wherein the control flow is a process comprising fewer threads than the 
number of concurrent connections. 

36. The virtual user of Claim 33, wherein the control flow is the main control flow of a process. 

37. The vi'rtuaf user of Clafm 33, wherein the pluraiily of connections are supported by the control flow. 

38. The virtual user of Qalm 33. further comprising at least one additional thread. 

39. A system for load testing servers, the system comprising: 

at least one client computer connected to at least one server under test through a computer network; 
a control flow executing on the at least one client computer and 

a plurality of concurrent connections to at least one server, wherein the plurality of connections are 
established by the control flow. 

40. The virtual user of Qaim 39, wherein flie control flow Is a fliread 

41. The virtual user of Claim 39. wherein flie control flow is a process comprising fiswer threads than the 
number of concurrent connections. 

4Z The.vlrtual user of Claim 39, wherein ^e control flow Is the nialn control flow of a process. 

43. The virtual user of Claim 39, wherein the pluraTrty of connections are supported by the control flow. 

44. The virtual user of aaim 39, further comprising at least one additional thread. 
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